The invention relates to a method for recording a scene using at least two time-of-flight cameras, which respectively comprise a light source and an image sensor, wherein image recording operations which comprise a measuring operation for determining depth information are carried out using each of the at least two time-of-flight cameras, wherein the measuring operation comprises the emission of modulated light by the light source, the detection of modulated light after the reflection on objects of the scene using the image sensor, and the calculation of depth information from the propagation time of the modulated light between the emission and the detection.
Time-of-flight (ToF) cameras concern cameras which not only record a 2D image, but also measure depth information for each recording pixel. Depth information is understood to be information on the distances between the individual objects of a scene and the ToF camera. ToF cameras are also known as active cameras because they are provided their own light source. The light emitted by said light source is reflected on objects of a scene to be recorded and thus reaches the detection region of the image sensor of the camera. The depth information is determined from the reflected light via propagation time or phase difference measurements.
The light sources are usually LEDs (light-emitting devices). Time-of-flight cameras emit modulated light. The light is typically OOK-modified (On-Off Keying) in the megahertz range (e.g. with 20 MHz) and thus emitted into the visual range of the own camera sensor. The reflected light components (photons) are recorded by the camera sensor and used for calculation of the distance of the reflected object. These depth data are then available for applications in addition to the greyscale image. Infrared light is used as illumination in most applications.
ToF cameras are widely used, especially in industry, in traffic telematics and in autonomous robotics. ToF cameras can be used for example in industry as filling-level or distance measuring devices in the close range up to 1 m. ToF cameras can be used as vehicle and person detectors and counters in the range of up to 30 m in traffic telematics. In autonomous robotics, ToF cameras can be used for recognising obstructions and for navigation.
The light of extraneous light sources can influence depth calculation. An image sensor in the form of a PMD chip (photonic mixer device) for example can perform background light suppression for each pixel if the background light does not have the same properties as the emitted modulated light of the camera. This function is typically used for suppressing sunlight or artificial illumination in rooms. In order to achieve suppression, the PMD chip records the light for each pixel when the own light source is switched on and off, and subtracts the two measurement results electronically. Correct suppression of background light is only possible if the intensity of the background light remains constant in all these periods and the PMD chip does not reach electrical saturation. In the case of artificial light sources which are modulated in the megahertz range the suppression does not work completely and the calculated depth data are erroneous in pixels.
Wrong depth calculations in a ToF camera occurs especially when the artificial light sources of other cameras are recorded in the visual range of the own camera sensor. The disturbing extraneous light can be recorded either directly or indirectly via reflections. In these cases, the results of the depth calculation are at least partly invalid.
Several cameras must be used for many fields of application, whose monitoring regions may overlap one another, e.g. in the monitoring of rooms, in traffic telematics, or in the control of several robots. The active light sources of the individual cameras respectively disturb the other cameras in the determination of the depth information because they not only detect the own light but also the light of other cameras, which is also known as extraneous light. These mutual disturbances lead to the consequence that the distance-related measuring results are distorted and the depth information no longer corresponds to the real conditions.
For the purpose of solving this problem, the individual cameras could be networked to each other with respect to control and function for parallel operation, which strongly increases the need for additional equipment however. In this case, the ToF cameras are operated in a network. In addition to data exchange, the network can also be used for synchronising image recordings in order to prevent disturbances of the cameras among each other. For this purpose, time stamps and reservation tokens are exchanged in the network. Only the camera that owns the reservation tokens is entitled to perform image recording. The token can be passed around or be administered centrally in a network server. These measures prevent that the measuring operations of two cameras influence each other during the recording of a scene. Precautions are thus taken in this solution that the modulated light of a camera is not emitted during the measuring or detection process of another camera.
Problems in the networking of ToF cameras according to this principle arise in such a way that this approach entails considerable additional costs of materials and development work. Furthermore, network latency can have a negative effect on the image throughput of the system as a result of the transfer duration of the reservation tokens. Similarly, this mechanism does not take the visual ranges of the cameras into account because a disturbance can only be expected in overlapping visual ranges.
Especially in applications in which the individual cameras move, a more or less complex mechanism would have to consider the continually changing overlapping visual ranges in order to maintain a high level of the image throughput with minimal disturbing effect.
The reliability of image recordings in networks also depends on the reliability of transferring the reservation tokens to all cameras.
In addition to these problems concerning data and control technology, the high costs for such equipment requirements concerning the connection, networking in a common signalling system, and the required complex software are often the reason that the mutual disturbances of the ToF cameras are not considered at all, which has a negative effect on the quality of the determined depth information.